In an ideal setup low latency may allow human-unnoticeable delays between an input being processed and the corresponding output providing real-time or near real-time characteristics. This can be especially important for trading services. In capital markets algorithmic trading may be used to process market updates and turn around orders within milliseconds. Trading using computers has developed to the point where millisecond improvements in network speeds offer a competitive advantage for financial institutions.
As such, congestion can be a serious issue, and there is a need to quickly determine when and why such congestion may occur. In some prior attempted solutions, a user may be able to attach a device to a network port and receive a packet dump. However, many networks operate with a traffic rate of nearly 10 Gbps. Such a traffic rate may lead to 1.25 GB of data created every second. A single day's traffic then requires 108 TB of storage. This approach wastes huge amounts of storage space and leads to unwieldy query times due to the amount of data.